{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Human-in-the-loop\n",
    "\n",
    "There are certain tools that we don't trust a model to execute on its own. One thing we can do in such situations is require human approval before the tool is invoked."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "We'll need to install the following packages:\n",
    "\n",
    "```bash\n",
    "npm install langchain @langchain/core @langchain/openai readline zod\n",
    "```\n",
    "\n",
    "We'll use `readline` to handle accepting input from the user."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LangSmith\n",
    "\n",
    "Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com/).\n",
    "\n",
    "Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces:\n",
    "\n",
    "\n",
    "```bash\n",
    "export LANGCHAIN_TRACING_V2=true\n",
    "export LANGCHAIN_API_KEY=YOUR_KEY\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chain\n",
    "\n",
    "Suppose we have the following (dummy) tools and tool-calling chain:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import { ChatOpenAI } from \"@langchain/openai\";\n",
    "import { Runnable, RunnableLambda, RunnablePassthrough } from \"@langchain/core/runnables\"\n",
    "import { StructuredTool } from \"@langchain/core/tools\";\n",
    "import { JsonOutputToolsParser } from \"langchain/output_parsers\";\n",
    "import { z } from \"zod\";\n",
    "\n",
    "class CountEmails extends StructuredTool {\n",
    "  schema = z.object({\n",
    "    lastNDays: z.number(),\n",
    "  })\n",
    "\n",
    "  name = \"count_emails\";\n",
    "\n",
    "  description = \"Count the number of emails sent in the last N days.\";\n",
    "\n",
    "  async _call(input: z.infer<typeof this.schema>): Promise<string> {\n",
    "    return (input.lastNDays * 2).toString();\n",
    "  }\n",
    "}\n",
    "\n",
    "class SendEmail extends StructuredTool {\n",
    "  schema = z.object({\n",
    "    message: z.string(),\n",
    "    recipient: z.string(),\n",
    "  })\n",
    "\n",
    "  name = \"send_email\";\n",
    "\n",
    "  description = \"Send an email.\";\n",
    "\n",
    "  async _call(input: z.infer<typeof this.schema>): Promise<string> {\n",
    "    return `Successfully sent email to ${input.recipient}`;\n",
    "  }\n",
    "}\n",
    "\n",
    "const tools = [new CountEmails(), new SendEmail()];"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "const model = new ChatOpenAI({\n",
    "  modelName: \"gpt-3.5-turbo\",\n",
    "  temperature: 0\n",
    "}).bind({\n",
    "  tools,\n",
    "});\n",
    "\n",
    "/**\n",
    " * Function for dynamically constructing the end of the chain based on the model-selected tool.\n",
    " */\n",
    "const callTool = (toolInvocation: Record<string, any>): Runnable => {\n",
    "  const toolMap: Record<string, StructuredTool> = tools.reduce((acc, tool) => {\n",
    "    acc[tool.name] = tool;\n",
    "    return acc;\n",
    "  }, {});\n",
    "  const tool = toolMap[toolInvocation.type];\n",
    "  return RunnablePassthrough.assign({ output: (input, config) => tool.invoke(input.args, config) });\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "// .map() allows us to apply a function to a list of inputs.\n",
    "const callToolList = new RunnableLambda({ func: callTool }).map();\n",
    "const chain = model.pipe(new JsonOutputToolsParser()).pipe(callToolList);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[ { type: \u001b[32m\"count_emails\"\u001b[39m, args: { lastNDays: \u001b[33m5\u001b[39m }, output: \u001b[32m\"10\"\u001b[39m } ]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "await chain.invoke(\"How many emails did I get in the last 5 days?\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding human approval\n",
    "\n",
    "We can add a simple human approval step to our `toolChain` function:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{=mdx}\n",
    "import CodeBlock from \"@theme/CodeBlock\";\n",
    "import HumanFeedback from \"@examples/use_cases/human_in_the_loop/accept-feedback.ts\";\n",
    "\n",
    "<CodeBlock language=\"typescript\">{HumanFeedback}</CodeBlock>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Examine the LangSmith traces from the code above [here](https://smith.langchain.com/public/aac711ff-b1a1-4fd7-a298-0f20909259b6/r) and [here](https://smith.langchain.com/public/7b35ee77-b369-4b95-af4f-b83510f9a93b/r)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Deno",
   "language": "typescript",
   "name": "deno"
  },
  "language_info": {
   "file_extension": ".ts",
   "mimetype": "text/x.typescript",
   "name": "typescript",
   "nb_converter": "script",
   "pygments_lexer": "typescript",
   "version": "5.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
